Learning from the past: Using the 2009 Learn and Serve cluster evaluation to develop the field's understanding of service‐learning implementation and outcomes
Paul Baumann, Director, Education Commission of the States [pbaumann@ecs.org]
Shelley Billig, Vice President, RMC Research Corporation [billig@rmcdenver.com]
Susan Abravanel, Vice President of Education, Youth Service America [sabravanel@ysa.org]
Therese Dary, Co‐Chair, National Coalition for Academic Service‐Learning [Therese.Dary@dpi.wi.gov]
Keywords: Learn and Serve America, quasi-experimental, K-12, multi-state study
Conference track: K‐12 civic and learning outcomes
Format: Symposium
Summary
This session will provide an overview of the evaluations of three 2009 Learn and Serve America (LSA) grantees: the National Center for Learning and Citizenship (NCLC), Youth Service America (YSA), and the Wisconsin Department of Public Instruction (DPI).
All participants in the cluster agreed to use a common quasi‐experimental evaluation design and a set of core common outcomes (academic engagement, academic performance, dropout prevention, and acquisition of 21st century skills) measured using the same survey scales. The research design allows for the aggregation of data and impact across programs and permits a rigorous exploration of various mediators, moderators, and covariates of outcomes. This session will provide an overview of three years of evaluation findings for three participants in the 2009 LSA cluster: NCLC, YSA, and the Wisconsin DPI. Participants represented approximately 20 states and included teachers, community members, and thousands of students across elementary, middle/junior high, and high school. Evaluations of state‐based service‐learning programs have generally not included the quasi‐experimental or experimental designs necessary to establish causality or determine effect sizes (Bradley, 2005). Further, few evaluations have included coordinated data collection strategies that are needed to establish more generalizable findings across various student populations (Bradley, 2005). The evaluations included in this symposium begin to address current shortcomings. Specifically, all of the studies incorporate a common, quasi‐experimental design including pre‐ and post‐test assessment, matched comparison groups, common survey measures, and common measures of demographics and program design characteristics. In addition, the studies collected achievement test scores and student performance data from all sites which enable researchers to understand the extent to which various program characteristics were associated with an effect on test scores and other high stakes assessments. Presenters will discuss the common outcomes examined and their underlying theoretical support; the study questions, methodology, and findings; and a discussion of why some findings were more robust than others. The presenters will discuss how implementation varied across the three organizations, and provide opportunities for session attendees to discuss the implications of evaluation findings and implementation strategies for future research and practice. References
Billig, S. H. (2004). Head, hearts, hands: The research on K-12 service-learning. In Growing to greatness: The state of service-learning project (pp. 12–25). St. Paul, MN: National Youth Leadership Council.
Bradley, L. R. (2005). K-12 service-learning impacts: A review of state-level studies of service-learning. In Growing to greatness 2005: The state of service-learning project (pp. 41–47). St. Paul, MN: National Youth Leadership Council.
Gould, J. (Ed.). (2011). The guardian of democracy: The civic mission of schools. Washington, DC: Campaign for the Civic Mission of Schools.
Pritchard, F. F., & Whitehead, G. I., III. (2004). Serve and learn: Implementing and evaluating service-learning in middle and high schools. Mahwah, NJ: Erlbaum.
To access materials from this session please click on the file link(s) below:
Paul Baumann, Director, Education Commission of the States [pbaumann@ecs.org]
Shelley Billig, Vice President, RMC Research Corporation [billig@rmcdenver.com]
Susan Abravanel, Vice President of Education, Youth Service America [sabravanel@ysa.org]
Therese Dary, Co‐Chair, National Coalition for Academic Service‐Learning [Therese.Dary@dpi.wi.gov]
Keywords: Learn and Serve America, quasi-experimental, K-12, multi-state study
Conference track: K‐12 civic and learning outcomes
Format: Symposium
Summary
This session will provide an overview of the evaluations of three 2009 Learn and Serve America (LSA) grantees: the National Center for Learning and Citizenship (NCLC), Youth Service America (YSA), and the Wisconsin Department of Public Instruction (DPI).
All participants in the cluster agreed to use a common quasi‐experimental evaluation design and a set of core common outcomes (academic engagement, academic performance, dropout prevention, and acquisition of 21st century skills) measured using the same survey scales. The research design allows for the aggregation of data and impact across programs and permits a rigorous exploration of various mediators, moderators, and covariates of outcomes. This session will provide an overview of three years of evaluation findings for three participants in the 2009 LSA cluster: NCLC, YSA, and the Wisconsin DPI. Participants represented approximately 20 states and included teachers, community members, and thousands of students across elementary, middle/junior high, and high school.
Evaluations of state‐based service‐learning programs have generally not included the quasi‐experimental or experimental designs necessary to establish causality or determine effect sizes (Bradley, 2005). Further, few evaluations have included coordinated data collection strategies that are needed to establish more generalizable findings across various student populations (Bradley, 2005). The evaluations included in this symposium begin to address current shortcomings. Specifically, all of the studies incorporate a common, quasi‐experimental design including pre‐ and post‐test assessment, matched comparison groups, common survey measures, and common measures of demographics and program design characteristics.
In addition, the studies collected achievement test scores and student performance data from all sites which enable researchers to understand the extent to which various program characteristics were associated with an effect on test scores and other high stakes assessments.
Presenters will discuss the common outcomes examined and their underlying theoretical support; the study questions, methodology, and findings; and a discussion of why some findings were more robust than others. The presenters will discuss how implementation varied across the three organizations, and provide opportunities for session attendees to discuss the implications of evaluation findings and implementation strategies for future research and practice.
References
Billig, S. H. (2004). Head, hearts, hands: The research on K-12 service-learning. In Growing to greatness: The state of service-learning project (pp. 12–25). St. Paul, MN: National Youth Leadership Council.
Bradley, L. R. (2005). K-12 service-learning impacts: A review of state-level studies of service-learning. In Growing to greatness 2005: The state of service-learning project (pp. 41–47). St. Paul, MN: National Youth Leadership Council.
Gould, J. (Ed.). (2011). The guardian of democracy: The civic mission of schools. Washington, DC: Campaign for the Civic Mission of Schools.
Pritchard, F. F., & Whitehead, G. I., III. (2004). Serve and learn: Implementing and evaluating service-learning in middle and high schools. Mahwah, NJ: Erlbaum.
To access materials from this session please click on the file link(s) below: